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Abstract 

In this work, the rate region of the vector Gaussian multiple description problem with individual 
and central quadratic distortion constraints is studied. In particular, an outer bound to the rate region of 
the L-description problem is derived. The bound is obtained by lower bounding a weighted sum rate 
for each supporting hyperplane of the rate region. The key idea is to introduce at most L-1 auxiliary 
random variables and further impose upon the variables a Markov structure according to the ordering 
of the description weights. This makes it possible to greatly simplify the derivation of the outer bound. 
In the scalar Gaussian case, the complete rate region is fully characterized by showing that the outer 
bound is tight. In this case, the optimal weighted sum rate for each supporting hyperplane is obtained 
by solving a single maximization problem. This contrasts with existing results, which require solving a 
min-max optimization problem. 
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I. Introduction 

Multiple description (MD) coding is a joint source-channel coding scheme aimed at combating 
unreliable communication links. The basic principle is to generate a set of descriptions for 
the information source with the property that any subset of the descriptions provides an 
approximation of the source with a certain fidelity. A natural research goal is to determine 
the transmission limits under some quality of service requirement. Generally, there are 2^ — 1 
distortion constraints for the L-description case, each corresponding to a particular combination 
of received descriptions. Thus, the problem complexity increases exponentially with an increasing 
number of descriptions. 

The pioneering work on the multiple description problem is the two-description achievable 
rate region by El Gamal and Cover [IJ (EGC scheme). The region was shown to be tight for a 
memory less Gaussian source by Ozarow |l2l, and was shown not to be tight in general when there 
is no excess rate (no redundancy) by Zhang and Berger This result was further extended 
to the L-description case in [4J. Later, the work in [I5]|,|l6l, provided an enlarged achievable rate 
region by using the random binning ideas from distributed source coding. 

The characterization of the (tight) rate region for the general L-description problem is quite 
challenging, and remains an open problem. Instead, the main focus has been on special cases, 
e.g., the case of symmetric side distortion constraints Q, or the case where only a subset of the 
distortion constraints is of concern [IHl, [|9l, [fTOl . 

For the particular case that only individual (only one description is received) and central 
receivers are of importance, Wang and Viswanath [BJ derived the optimal sum rate for the vector 
Gaussian source. In [10|, Chen considered the same transmission scenario, and derived the rate 
region. However, his work was limited to the scalar Gaussian source. The direct extension of 
his work to the vector Gaussian source appears to be difficult if possible. 

The present work considers the rate region of the vector Gaussian multiple description problem 
with individual and central distortion constraints. One major contribution is that an outer bound 
is derived for the considered rate region. The outer bound is formulated by lower-bounding a 
weighted sum rate associated with a supporting hyper-plane of the rate region. The expression 
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for the bound only involves a maximization process. 

Another contribution is that when the new approach is applied to the scalar Gaussian multiple 
description problem, the derived outer bound is shown to be tight, thus fully characterizing the 
rate region. Note that the expression for the bound corresponds to a maximization problem as 
compared to that in [ fTO| which involves a min-max game (or a max-min game), thus significantly 
reducing the problem complexity. 

We now summarize the notations used in this paper. Lower case letters denote scalar random 
variables and boldface lower case letters denote vector random variables. Boldface upper case 
letters are used to denote matrices. Specifically, we use and / to denote the all-zero matrices 
(including zero vectors) and the identity matrices, respectively. We also use H to denote all-one 
matrices. The operator | ■ | refers to the determinant of a matrix, unless otherwise specified. The 
operator E(-) denotes the expectation. For random vectors yi and we use E [^11^2] to denote 
the conditional expectation of given ^/2, and Cov\yi\y2\ to denote the covariance matrix of 
7/1 —E [?/i|t/2]- The partial order >- {>z) denotes positive definite (semidefinite) ordering: A)~ B 
{A y B) means that A — S is a positive definite (semidefinite) matrix. All the logarithm 
functions are to base e. 

II. Problem Formulation and Main Results 

In this section we first define the multiple description rate-region problem formally. We 
then introduce the Gaussian multiple description scheme for the problem, which provides an 
achievable rate region. Finally we present the main results of the paper. 

A. Problem Formulation 

Suppose the information source is an i.i.d. process {a;[m]} with marginal distribution 
A/^(0, Kx), i.e., a collection of i.i.d. real Gaussian random vectors. Let the covariance matrix 
be an X positive definite matrix. There are L encoding functions at the transmitter, each 
encoding a source sequence of length n, x"- = {x[lY, . . . , x[nY)\ Denote the resulting codewords 
as //"^(x"), / = Let Cl'^\ I = denote the corresponding codebooks, i.e., 
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//"^(x") G Ci^\ The output of the /'th encoder is sent through the Vth communication channel 
at rate Ri = ^ log |C;''"^|, where |C/"''| denotes the cardinality of the codebook. 

In response to the L descriptions, there are L individual receivers and one central receiver. 
The /'th individual receiver uses its received codeword to generate an approximation gi{fl^\x'^)) 
to the source sequence £c", / = 1, . . . , L. On the other hand, the central receiver generates an 
approximation to the source sequence based on all L codewords. Since we are only interested 
in the quadratic distortion measure, the optimal approximation is given by the minimal mean- 
squared error (MMSE) estimation of the source sequence. We say a rate vector . . . , Ri) is 
achievable if there exist, for all sufficiently large n, encoders of rates . . . , Rl) and decoders, 
such that 



1 " r 

- Cov a;[m]l//"\a;") 



n 

m— 1 



<Di, 1 = 1,. ..,L, (1) 



n 

m— 1 



^ Do- (2) 



The rate region TZ(K,j., Di, . . . , D^, Dq) is the convex hull of the set of all achievable rate 
vectors subject to the individual side distortion constraints Di, I = 1,...,L, and the central 
distortion constraint Dq. Throughout the paper, we consider -< Dq -< Di, V/ E {1, . . . , L}. 

We focus on characterizing the rate region 'R{Kx, Di, Dq). Our work is an extension 

of [IHl which only addressed the optimal sum-rate problem for the above transmission scenario. 
There are two motivations behind our work. First obtaining the rate region is of great interest 
from a theoretic point of view. Second the rate-region expression can provide insight in designing 
efficient asymmetric (i.e, the rates are different across the channels) multiple description systems. 

Motivated by the fact that TZ^K^, Di, . . . , D^, Dq) is a closed convex set, we consider 
characterizing its supporting hyperplane, which can be formulated as an optimization problem 
of the form: 

L 

min V/3;i?;, (3) 

{Ri,...,Rl)&TI{K,,Di,...,Dl,Do) 

where (3i, I = 1, . . . ,L, are arbitrary positive parameters. The case where at least one of the 
weighting factors is zero defines a trivial problem. To see this, notice that one can always put 
enough rate to the descriptions associated with the zero weighting factors, in order to meet the 
central distortion constraint. Then one simply needs to use the minimal single-description rate 
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for each remaining description in order to satisfy the individual distortion constraints and thereby 

achieve the minimal (optimal) weighted sum rate. 

Without loss of generality, we may assume /3i > . . . > /3l > 0, which can always be obtained 
by rearranging the description indices. It may happen that some weighting factors are equal. We 
further group the weighting factors {(32, (^2, /Ss, . . . , (the first element is modified on purpose, 
and it will be explained later) according to their values. The elements within each group have 
the same value and different groups take different values. Denote the resulting weighting factors 
as ai > a;2 > . . . > aj, where aj is associated with rrij rates in ([3]). With this, it follows that 
Sj=i = L, and rrii > 2. The maximum achievable value of J is L — 1. Denote 

i 

Mi=Y,m, j = l,...,J (4) 

i=l 

and let M° = 0. The optimization problem ([3]) can be rewritten as 

min ao^i + "j \ ' 

(Ru--;RL)(in(K^,Di,...,DL,Da) ^ +V 

j=l 1=1 

where = /3i — /32- The reason to distinguish different weighting factors is to facilitate the 
derivation of the outer bound to ([3]). It will be shown later that the choice of is of little 
importance. It doest not affect the construction of the outer bound. 

B. Gaussian Description Scheme 

In this subsection, we introduce the Gaussian description scheme, with which an achievable 
rate region is then obtained. Conceptually, in the transmission of a vector Gaussian source 
X, L parallel noisy sub-channels with additive Gaussian noises are constructed. This has the 
advantage that the transmission distortions and rates can be easily analyzed. Let Wi, . . ., wl 
be A^-dimensional Gaussian vectors independent of x, of which the marginal distributions are 
denoted as A/'(0, Ki), I = 1, . . . , L. Define 

ui^x + wi, l^l,...,L. (6) 

Such a channel is referred to as a Gaussian test channel [fTT|. For any subset S* C {1, . . . , L}, 
we use Ks to denote the covariance matrix of all wi, I E S. With a slight abuse of notation, 
we refer to ... j^} as 
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We consider the MMSE estimation of x using subsets of Ui, I = 1, L. It can be shown 
that 

Cov[x\ui,l e 5] - {K-' + {In, • ■ • , In)Ks\In, ^iv)*)"' , V5 C {1, . . . , L}. (7) 

Define 

Ks^[{In,...,In)Ks\In,...,In)T' ^ 5C{1,...,L}. (8) 

It is immediate that K^i = Ki, I = 1,...,L. Similarly to the introduction of K^, we let 
K ^. = To clarify, for each S C with cardinality \S\, Ks is of size 

N\S\ X N\S\, whereas Kg is of size N x N. Plugging ^ into © produces 

Coy[x\ui,leS]^{K-'+Ksy\ SC{1,...,L}. (9) 

We can define a virtual additive Gaussian noisy channel for each subset S in where the 
covariance matrix of the Gaussian noise takes the form K Therefore, the quadratic distortion 
of the MMSE estimation of the source x from this channel output gives the same expression as 
®. 

The achievable rate vector by using the Gaussian description scheme has been studied in detail 
in flU (for the scalar source case), and in ^ (for the vector source case). We summarize the 
result in the following lemma. 

Lemma 2.1 (SEIjl): For every such that 

Cov[a;|M,] ^ Di, I = l,...,L 

Co\[x\ui, . . . ,ul] di Do, (10) 



the rate vector satisfying 

s 

is achievable. 



.les 



hiui,l e S\x) ^ llog ^'^^^l^^ ^ V5C {!,...,£}, (11) 



Lemma 12.11 defines an achievable rate region. In Section |Vl we will show that the achievable 
region is tight for a scalar Gaussian source. 
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C. Outer Bound to the Rate Region 

In this subsection we present our new outer bound to the rate region TZ{Kx, -Di, . . . , Dl, -Do)- 
The expression of the outer bound is obtained by lower-bounding a weighted sum rate ^ of 
every supporting hyper-plane of the rate region. 

In our construction of the outer bound in our work, we introduce a set of auxiliary Gaussian 
random vectors as in [jlOll . Differently from IITOll . we impose a Markov structure on those auxiliary 
random vectors according to the order of the weighting factors. The main reason to group the 
weighting factors (/32, /32, /^s, ■ ■ ■ (see Subsection III-A|) by their values is to identify the 
number of auxiliary Gaussian random vectors needed for the construction of the outer bound. 
For the optimization problem in ([5]), we introduce J auxiliary Gaussian random vectors, Zj, 
j = 1, . . . , J, one for each distinct weighting factor (i.e., Zj is associated with aj.). When all 
the weighting factors are identical (i.e, J = 1), only one auxiliary random vector is needed. This 
special case is for deriving an outer bound to the sum-rate, which was addressed in 

We now explain the Markov structure of the J random vectors in more detail. The covariance 
matrices of the auxiliary vectors are chosen such that a high-indexed vector has a large covariance 
matrix. Denote the covariance matrix of Zj as Nj, j = 1, . . . , J. The Markov structure can be 
mathematically formulated as -< aiNi -< «2-^2 -<•..-< ajNj. By using this partial ordering 
structure, the derivation of the outer bound is significantly simplified. Further, it facilitates the 
recognition of optimality conditions for the outer bound to be tight. On the other hand, the work 
in [fTOl uses L auxiliary random variables without imposing a Markov structure on them. 

Considering the multiple description coding with respect to individual and central distortion 
constraints for a Gaussian source with distribution J\f{0, K^), we have the following outer bound 
to any supporting hyperplane of the rate region. 

Theorem 2.2: Given the distortion constraints (Di, . . . , D^, Dq) and a zero-mean vector 
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Gaussian source with covariance matrix K^, the weighted sum rate © is lower-bounded by 



J / rrij \ 

+i 



j=i \i=i / 
> sup — log H log 



1 

+ 2 °S ay\Do\U:=[\Nj + D,,.-.^^\\Nj-Nj^,\ j' ^ ^ 

where B{{Ni}f^^) = {{Ni,...,Nj) G M^x^-^|0 ^ aiA^i ^ asA^s ^ ... ^ ajNj}, and 
J > 1. For the special case that J = 1, the lower bound is expressed as 

«o, \K.\ a. ( \K^\\K^ + N^\^-^\D^ + N,\ \ 
aoRi + ai y Hi > sup — log t-z-— + — log j . (13) 



Proof: See Section HIl ■ 
One special case of B({Ni}f^-i) in (fT2)) is of particular interest. Letting Ni = [e/ai]lN, 
Nj = [e(l + e + . . . , +e^''^)/aj]lN, j = 2, . . . , J, and e ^ 0+, we obtain from ^ the 
following lower bound 

««..|:„,g«,-,.).^,„s.^.i:fg,„s.^). 04, 

This bound is actually the weighted summation of the L minimum single-description rates, 
^ log i = 1, ■ ■ ■ ,L. Note that the central distortion constraint is not involved in the 
bound. For the extreme case that the central distortion constraint is loose, ([14)) becomes 
a tight bound. In this situation, any hyperplane attains the rate region at the corner point 

D. Optimal Weighted Sum Rate 

In this subsection, we first present the optimality conditions for the Gaussian description 
scheme to produce the optimal weighted sum rate. Specifically, we derive the optimality 
conditions on the covariance matrix K,^^j as introduced in subsection III-B[ Then we discuss 
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how to compute the optimal matrix if it exists. 

For the application of the Gaussian description scheme, we provide optimality conditions under 
which the performance of the scheme reaches the outer bound (fT2l) . Therefore, if a Gaussian 
description scheme can be constructed for a weighting vector (ao, ai, . . . , aj) such that the 
optimality conditions are satisfied, the corresponding optimal weighted sum rate can be obtained. 

Theorem 2.3: If, for any given K,j., there exists a such that: 

1) Layered correlation: 

= -Aj, y 1 < k < M(^^ +i, i = I,.. .,mj, j ^ I,. . .,J (15) 

2) Proportionality: 

aj (^Aj + K^^ j^j,^'^ = "j+i (^Aj+i + K^^^ ^^j,^^'^ , j = 1,..., J-1, 

(16) 



E 



where -< aiAi -< OL2A2 -<...-< ajAj -< ajK^, and such that the set of distortion constraints 
(Di, . . . , Dl, Do) is achieved with equality, then the outer bound given by (fT2l) is tight. 

Proof: See Section JVl ■ 
The property of layered correlation in Theorem 12.31 refers to the fact that the correlation 
matrix of wj and any lower indexed random vector k < j remains the same. The number 
of different correlation matrices is determined by the number of different weighting factors. For 
the optimization problem ([5]), J different correlation matrices have to be constructed. Informally, 
each correlation matrix Aj in (fTSl ) controls the redundancy among the nij descriptions associated 
with aj, and the set of descriptions associated with {ai,i < j}. Further, a small correlation 
matrix corresponds to a high redundancy among the associated descriptions. For the extreme 
case that the central distortion constraint is loose, the optimal correlation matrices take the form 
Ai = ■ ■ ■ = Aj = 0. The L descriptions are highly correlated, implying a high redundancy 
embedded in the descriptions. The optimal weighted sum rate for this special case is given by 
d). 

The proportionality condition (fT6l ) imposes an ordered structure on the correlation matrices, 
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i.e., -< «! Ai -< ^2 A2 -< . . . ^ CijAj -< ajKx- This ordered structure is essentially determined 
by the ordered structure on the weighting factors, i.e., ai > 02 > ■ ■ ■ > «j. We now explain 
this property in an informal way. If the central distortion constraint is active, each description 
has to carry an extra rate besides the minimum single-description rate to account for the central 
distortion constraint. Since a large weighting factor enforces a large penalty on the associated 
description rate in ©, it is desirable that the descriptions with a large weighting factor carry a 
small amount of extra rates. By relating a small correlation matrix with a large weighting factor 
(i.e., Aj corresponds to a^, j = 1, . . . , J), the redundancy introduced by the correlation matrix 
is large, rendering a small amount of extra rate for the descriptions associated with the large 
weighting factor. (fT6l) fully characterizes the relationship between the correlation matrices and 
the weighting factors. 

Next we consider how to obtain the the covariance matrix K^j satisfying the conditions in 
Theorem 12.31 if it exists. We first study the property of the layered correlation ([151) . We find 
that if the covariance matrix Kyj has the layered structure (fT5l) . the matrices and Aj, 

j = 1, . . . , J carry some simple relationships. We present the result in the following lemma. 

Lemma 2.4: Suppose the covariance matrix Ky^ satisfies (flSl) . If Ky^ >- and Aj y 0, 
j = 1,..., J, then 



Kn_Mi} + ^1) = ETJi (K.. + A,)-' ^^^^ 
(^{1 Ain + A,) ' = (^{i,...,Mr^} + ^^■) + ^TJi {KMr+^ + A,y\ j = 2, . . . , j 



Proof: See Appendix O ■ 
Equ. (fT6l) together with (flTI) fully characterizes the relationships among ^^^^ and Aj, 

j = 1, . . . , J. Note that given {Ki : / = 1, . . . , L}, (fT6l)-(fT7]) can be interpreted as a set of 
functions Aj,j = 2, . . . , J, and Mf}^^ ~ 1, . . . , J over Ai. In other words, Ai is the 

only free covariance matrix under (fT6l)-(fT7]). The problem for obtaining K.^, is now reduced to 
determine the correlation matrix Ai and {Ki : / = 1, . . . , L}. In fact, the covariance matrices 
{Ki : I = 1, . . . , L} can be determined by the individual side distortion constraints. On the other 
hand, the correlation matrix Ai can be determined by the central distortion constraint. 
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We now revisit the side and central distortion constraints. Suppose the distortion constraints 
(Di, . . . , Dl, Do) are achieved with equality using the Gaussian description scheme. By 
applying we have 

Ki^{D^' -K-Y\ l^l,...,L 

K^ = {D^' -K-^)-\ (18) 

If there exists a positive definite solution Ai to (fT6l)-(fT8l). then the distortion constraints are met 
with equality. The remaining work is to check if the matrix K^, constructed from the solutions 
of (fT6l)-(fT8l) is positive definite. It turns out that as long as Ai is a solution to (fT6l) -(fT7l) where 
{Ki >- : / = 1, . . . , L} and K ^, y are constant matrices, the resulting is always positive 
definite. We state this formally below. 

Lemma 2.5: Let y and Ki y 0, I = 1, . . . , L he constant matrices in (fT6l)-(fT7]). If for 
some Ai y 0, (fT6l)- (fr7l) are true and -< aiAi -< 0:2^2 -<...-< ajAj, then the matrix Kyj 
constructed using (fTSi) is positive definite. 

Proof: See Appendix El ■ 

We summarize the result in the following proposition. 

Proposition 2.6: Given distortion constraints (-Di, . . . , Dl, Dq). If there exists a solution A\ 
to (fT6l)-(fT8]). then the Gaussian description scheme with constructed using ([T?] ) achieves the 
optimal weighted sum rate. The optimal N j in (fT2l) is given as N j = (Aj(A^)^^ — K~'^)^^, 
j = l,...,J. 



E. Rate Region for the Scalar Gaussian Source 

In this subsection, we use Theorem 2.3 to prove that for the scalar Gaussian case, the outer 
bound on the weighted sum-rate provided by (fT2l) is tight. This completely characterizes the rate 
region for the scalar Gaussian case and parallels a recent result of Chen [flOl . 

Theorem 2.7: For a scalar Gaussian source and individual and central distortion constraints, 
the complete rate region is given by (fT2l) and (fT3l) . 

Proof: See Section |Vl ■ 
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Remark 2.8: It should be noted from Theorem 12.71 and (fT2l)-(fT3l). that the scalar Gaussian 
rate region is obtained by solving a single maximization problem, whereas the approach of [lOl 
requires the solution of a min-max optimization problem. 



III. Proof of Theorem 12.21 

Before presenting the proof of Theorem 2.2, we need the following lemma: 
Lemma 3.1: Assume that v and x" are arbitrarily distributed random variables (which might 
be correlated) and assume that given v, a?" has a density. Denote the dimensionality of a?" 
as nN. Let {zifm]}"^^^ and {z2[m]}J^^]^ be two i.i.d. random Gaussian vector processes with 
marginal distributions J\f(0,Ni) and J\f(0,N2), respectively. Both Ni and N2 are of size 
N X N. Let = [zi[l]*, . . . , Zi[ri]*]*. In a similar manner, we can define Zg. Both z" and Z2 
are independent of v and x". Let iii > fi2 > and -< /UiA/"i -< ^2^2- Then there is 

where the equality holds if aj" and v are jointly Gaussian and satisfy 

Cov[a;'>] = 7„ ® (^i - /i2)A^2(M2A^2 - MiA^O^'A^i, (20) 

where ® denotes the Kronecker product [fT2ll . 

Proof: See Appendix El ■ 

When the problem specializes to the scalar source case (i.e. N = 1), an alternative upper 
bound of the right hand side of (fT9l) was derived by Chen IfTOl . The main difference is that 
the upper bound in [fTOl requires solving an optimization problem, and does not admit a simple 
closed-form expression. 

We now proceed to present the argument for the outer bound. We first consider the case J > 1. 
We define J i.i.d. random Gaussian processes z" = {^i["^]}m=i ^^^^ marginal distributions 
A/'(0, Nj), j = 1, . . . , J, respectively. Let Ci = //"^(as"), / = 1, . . . , L, be the discrete random 
variables. Suppose that z", • • ., z} are independent of a?" and Ci, I = 1, . . . , L. Next we introduce 
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J processes t/" = (yJl]*, . . . , vAnff, j = 1, . . . , J by 



yAm] = x[m\ + Zj[m]^ m — 1, . . . ,n and j = 1, . . . , J. 



(21) 



It follows that each sequence {yj [m]} is an i.i.d. process with marginal distribution M{0, Ky.), 
where Ky^ = + N j, j = 1, . . . , J. The following sequence of inequalities provides a lower 
bound to the weighted sum rate ([5]): 



3 = 1 \i=l 

J 



,7 / rrij 



1=1 ) 

3 = 1 i=l 

J 

,7 »rij 

= ao Hx-) ~ + E E (My") - (y"|c^Mr^+.)) + 

3 = 1 1=1 

J 

^ (y^_i|Ci, . . . , - ajh . . . , - - aj)/i (a;"|Ci, . . . , 

J=2 



- aj [h (y}|Ci, . . . , Cl) - (a;"|Ci, . . . , Cl)] + aM^'') ~ aMVi )- 



(22) 



The construction of the inequality (a) is crucial in the outer-bound derivation. For each distinct 
weighting factor, we essentially introduce an auxiliary Gaussian process. 



We now consider deriving a lower bound to (|22l) . As a;" and 2/", j = 1, . . . , L are Gaussian 
vectors, we easily obtain 

h{x^) = Uog{2nef^\K,\^, 

/i(y^") = ilog(27re)^"|K, + Ar,|", j = l,...,J. (23) 
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We will also use some entropy-related inequalities developed in JSl, which are 
1 

iog( 

1 



/i(a;"|Ci) < -log(2^er"|Di|", (24) 



h{y]\CMr+i) < 2 log(27re)^^"|r>^,.-i^, + AT,!", * = 1, . . . , m, and j = 1, . . . , J, (25) 
h {y^j\C^, ...,CL)-h (a^"|Ci, ...,Cl)>^ log '^"^^-^' . (26) 

By using Lemma I3.1[ the remaining quantities in (l22l) can be lower-bounded as 

aj-ih (y]_i\Ci, . . . ,Cj^,^,-i) - ajh (y]\Ci, . . . ,C^,^,-i) - {a^^i - aj)h (a;"|Ci, . . . ,Cj^,^,-i) 

- 2 l°S«f_JAr,-iV,_i|+ 2 «f|iV,-Ar,_i| ' ^-2,..., J, (27) 

where -< ai A/"i -<...-< ajNj. The equalities hold in (|271) if Ci, . . . , and a;" are jointly 
Gaussian with conditional covariance matrices 

Cov[a;"|Ci, . . . , = /„ ® - ajOiVj^-A^j - a.^^N N j = 2,...,J. (28) 

Plugging (|23])-(|271) into (l22l) produces the expression (fT2l) . 

For the case J = 1 (i.e., the sum-rate case), we can follow the same derivation steps as those 
for J > 1. The argument for this special case is actually the same as that in |f8l|. An alternative 
derivation is to start with the expression of the outer bound for J > 1, and let a2,---,aj 
approach ai in the expression. The proof is complete. 

IV. Proof of Theorem 12.31 

In this section we study the optimality of the Gaussian description scheme in achieving the 

outer bound (fT2l) . We identify the optimality conditions by investigating the inequalities used in 

the outer-bound derivation under the Gaussian description scheme. 

Similarly to the derivation of the outer bound, we introduce J auxiliary Gaussian random 
vectors zj ~ J^{0, Nj), j = 1, . . . , J, independent of x and all Wi's. As for (|271) . we put a 
partial ordering constraint on Nj, / = 1, . . . , J: 

-< aiNi -<...-< ajNj. (29) 

To simplify the discussion, we introduce aj+i = 0. Letting yj = x + Zj, j = 1, . . . , J, we 
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obtain the following lower bound to ([5]) under the Gaussian description scheme: 

J I raj \ 

j=l / 



j=i \i=i 

J /Ml 

= ao ih{ui) - h{ui\x)) + ^(a^ - a-j+i) '^h{ui) - h (ui, . . ■,Uj^jj 

3 = 1 \t=l 

J J / "ij 

j=i i=i 
(6) •■' 

nij 

- h (ui, . . . , + X ("A/r'+'l^j) " ("1' ■ • • ' "A^j- IVj 

1=1 

,7 / \ 

= ao/(Mi;a;) + ^aj I ^ (■"Afr'+'' ^-j) ) ^ [^(^^l^i' • ■ • ''"i) ^ ■ • ■ 

J = l \i=l ^ / 

J 

+ X ["j-i'i (i/j-ilwi) • • • > ■"A/r') ^ I'"!' • • • ' "Afri) " ~ "j)^ (a^l^i, • • ■ ,uj^q- 

+ aih{x) - aih{y^), (30) 
where step (a) follows from Lemma 12.11 

We now study the inequality (b) in the derivation of (|30l ). It is straightforward that if 

h (ui, . . . ,u^,jj-i\yj'j + ^^{'^Ml-^+t\yj) - ^ (■"!' • • • '■"a/^ =0' j = ^,---,J, (31) 

i=l 

then (b) is actually an equality. The fact that (|3T1) hold implies that conditioned on y^, 
it^i-i^^, ■ ■ ■ , Uj^^p, and [u\, . . . , t^\q-iY independent. In other words, the rnj descriptions 
associated with aj, and the set of descriptions associated with {ai,i < j}, are conditionally 
independent given yj. Further, the conditional independencies (|3T1) exhibit a layered structure. 
The conditional independency w.r.t. y^ only refers to those descriptions with weighting factors 
no larger than aj. The number of layered conditional independency is determined by the number 
of distinct weighting factors. Note that all the random variables involved in (l30l) are Gaussian 
vectors. The condition (|3T1) can be equivalently described by constraints on their covariance 
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matrices. We present these constraints in the following proposition. 

Proposition 4.1: There exist a set of matrices (Ni, . . . , Nj) such that (|29|) holds and (|3TI) is 
true if the covariance matrix K^^, satisfies the following three conditions 

• equation (fTSl ) is true; 

• the covariance matrix Nj takes the form 

N, = {AJ' -K-Y\ j = l,...,J. (32) 

• The correlation matrices Aj, j = 1, . . . , J satisfy a partial ordering constraint: 

-< aiAi -< 0:2^2 -< . . . ^ ajAj -< ajK^. (33) 



Proof: 

We first derive the conditions such that (|3T] ) is true. Then we consider the partial ordering 

constraint (|29l ) to obtain the additional conditions. 

Note that the J conditional independencies in (|3T| ) have the same structure. We can focus on 
one particular case. The argument for other cases are the same. For a particular j, we use similar 
derivation steps as in the proof of (81 Proposition 2] to obtain the conditions. We find that ( |3T| ) 
holds for each j when 



Aj, VI < fc < M(~'^ +i, i = 1, 



E 

-< Aj -< Kx 

N,={A-'-K-')-\ 

Proposition 2 in [[81 actually considered the conditional independency for J = 1 (the sum-rate 
case). 

Next we show that (|32l) and (|33l) together are sufficient to produce (|29l ). By using Lemma 
|A.2[ we have 
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^ "j-^i -< aj+iNj+i, j = 1, . . . , J - 1. 
The proof is complete. ■ 

We consider the matrix satisfying the conditions in Proposition |4.1[ We proceed to 
recognize the additional conditions such that (l30l) reaches the outer bound (fT2l) . Again since all 
the random variables involved in (|30l) are Gaussian vectors, knowing their covariance matrices 
is sufficient to characterize the expression. We list the distortion conditions as 

Cov[a;|M,] = D,, l = l,...,L, (34) 



Cov 



X\Ui, 



= iK-' + ^^ Ajl,- "^-^ AtY\ J =2,..., J, (35) 



Cov[x\ui, . . . ,ul] ^ Dq. (36) 



The above distortion conditions are derived from (I23l)-(l28l) and (1321) . In particular, (1351) is obtained 
by combining (|28l) and (|32l ). (|34l ) and (|36l ) are for the individual side distortion constraints and 
the central distortion constraint, respectively. They correspond to (|24|)-(|261). By using (|32|) and 
(l34l)- (|36l) . it can be shown that (l30l) gives the same expression as the outer bound (fT2)) . 



It is now clear that (fT5l) . (|33] ) and (|34l)-(|36l) are optimality conditions on the covariance matrix 
for the weighted sum rate. When a proper can be constructed satisfying the optimality 
conditions, the outer bound (fT2l) is tight under the expression (|32l ). From the condition ( |35] ) 
can be rewritten as 

K-^' ^^7+1' J = l,-.-,J-l. (37) 

By using Lemma lA.ll on dTT] ). we then arrive at the proportionality condition (fT6] ). For each 
j = l,...,J— 1, the expressions (fT6l) and (|37l) are equivalent when Aj and Aj+i are positive 
definite. However, the condition (fT6l) is more powerful in that it can handle the case that Aj y 0, 
j = 1, . . . , J. The generality of (fT6l ) over (|37l ) might be useful to extend Theorem 12.31 to the 
case that the central distortion is loose. In our work, we will not discuss this case. The proof is 
complete. 
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V. Rate region for Scalar Gaussian Source 

In this section we consider the rate region of the multiple description problem with individual 
and central receivers for a scalar Gaussian source x ~ J\f(0,al). Denote the individual and 
central distortion constraints as di, I = 1, . . . , L, and do, where < do < di < for all 
I = 1, . . . , L. We construct the Gaussian test channel with wi, I = 1, . . . , L, such that 



ki = Gov \w{\ , 



E 



= -cr| \fl<k<M(^+i, i = l,...,mj, and j = 1,..., J. (38) 



The correlation coefficient cr| corresponds to Aj in ([T?] ) defined for general vector case. With 

this Gaussian description scheme, we find that the outer bound in Theorem 12.21 is tight. 

It is evident from Proposition 12.61 that the conditions (fT6l) and (fTTI) play an important role in 

establishing optimality of a Gaussian test channel. Note that for the scalar case, (fT6l ) is equivalent 

to (|37] ) when the correlation coefficients cr| > 0, j = 1, . . . , J. We study the properties of 

j = 2, . . . , J and k^-^ j = 1, . . . , J as functions of aj through (l37l) and (fTTI) . 

Lemma 5.1: Let ki > 0, I = 1, . . . , L, he constants. Define (t|, j = 2, . . . , J, and k^-^ j^qy 
j = 1, . . . , J as functions of af, expressed as 



{k{l,....Ml} + = Er=l i'^l + 



(39) 



and fc-i =- '^2+^—^-- , j = l,...,J-l. (40) 

Then there exists af > such that the variables af, I = 2, . . . , L, are monotonically increasing 
over G (0, af), and the variables k^-^^ j^py, j = 1, . . . , J, are monotonically deceasing over 
af e (0, (xf). For any af G (0, af), there is 

< aifTi < a2cr| <,...,< ajCTj. (41) 

Further, 



crj crj_i as o-^ ^ 0, where j ^2,. ..,J (42) 
^-^(E^^') asa?^0, (43) 
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(44) 



Proof: See Appendix |D] for the proof. ■ 

Upon establishing the result in Lemma I5.1[ we are now ready to argue that for the scalar 
Gaussian source, the multiple description scheme using Gaussian descriptions achieves the outer 
bound in Theorem \12\ In particular, we will show that when either the central distortion constraint 
or the side distortion constraint is loose, the outer bound (fT2l)-(fT3l) is still tight. Let 

h = {dY^ - a-^)-\ l^l,...,L (45) 

in Lemma 15. 1[ To simplify the derivation, we use k^{crl) and crj(cr^), j = 2, . . . , J to denote 
the functions of and crj, j = 2, . . . , J over af. From (l43l) . we denote the upper bound of 
k^{crf) as k^ = k^^^ . Considering the central distortion constraint, we introduce 

hi ^ {do' -a-')-'- 

We now consider two scenarios depending on the relationship between and k^: 

Scenario 1 (k^ > k^): The analysis for this scenario is trivial. It corresponds to the case 
that the central distortion constraint is loose. By letting 

the optimal Gaussian test channel is thus specified. The resulting cental distortion is 

L 

'^s = E'^r'-(i-i)'^."'- (46) 

i=l 

Further, there is c?q <= (Iq. In this situation, the optimal rate for each channel is actually the 

minimum single description rate, i.e. Ri = | log ^, = 1, . . . ,L. From (fT4l) . it is straightforward 

that the Gaussian description scheme achieves the optimal weighted sum rate. 

Scenario 2 (k^ < k^): From Lemma [STl it is ensured that there exists > such that the 
corresponding parameter fc^(cr^) satisfies 
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In this situation, there are two possible outcomes when comparing (Jj{(Tf) with a^. 
We first consider the case that crj{d-f) < a^. From Lemma I5.1[ it follows that 

< aia\ < Q!20'|((Ti) <,...,< aj(CTj) < aja^. 

We construct the Gaussian test channel using (af, . . . , aj{af)) and (|45l) . The corresponding 
multiple description scheme using Gaussian descriptions satisfies the conditions of Theorem \2.3\ 
Thus, we obtain the optimal weighted sum rate. 

Next we consider the case that cTjicrf) > a^. This particular case actually corresponds to the 
situation that the individual distortion constraints associated with the weighting factor aj are 
loose. To show that the outer bound (fT2l) is still tight, the basic idea is to first construct a new d']^ 
smaller than or equal to in P3] ). and then consider the new multiple description problem with 
distortion constraints {di, . . . , di-i, d'j^, do). The key observation is that the resulting optimal 
sum rate (fT2l) of the new multiple description problem is only a function of {do, . . . , di-i, do), 
and is unrelated with d'^ or di- From Lemma [5TT1 there exists < af < af such that 

ki<kjal) 

In this situation, we have 

-1 -1 '"J _i 

i=l 

where A > 0, and k^^ m-^-'^} ^ function of af as defined in Lemma 15.11 We use the 
enhancement technique WSl to find a /e^ such that fc^ < (correspondingly d'j^ < di). Letting 

and using the fact that a1 = aj^af), we arrive at 

k'L^[ikL+al)-'+\]-' -al (48) 

The verification of A;^ < k^ is straightforward. Equ. (l47l) can be rewritten in terms of k'^ as 
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From Lemma 12.51 and Lemma |5.1[ we conclude that the Gaussian test channel 
built from {ki, . . . ,kL-i,k'^) and {af, . . . , aj{aj)) is valid. The resulting distortions are 

(di, di-i, d'^, do), where < d'^ < di- 

We now show that the optimal weighted sum rate (fT2l) of the above Gaussian description 
scheme is unrelated with d'j^. The idea is to construct a sequence rfo(e) such that (j}(a^(e)) < 
o1, thus allowing the application of Theorem 12.31 Then by letting e — )■ 0, the argument is 
straightforward. Let 

d^{e) = {{k:, + e)-^+alr\ (50) 

where e is chosen such that e > and (A;^ + e)"^ > Yl!i=i + ^zT^- Taking {ki, . . . , ki-i, k'l) 
in Lemma 15. 1[ we obtain 

Considering the multiple description problem with distortion constraints (rfi, . . . , d'l^^ do{e)), the 
situation corresponds to the one discussed in the first case. As long as e 7^ 0, the optimal 
weighted sum rate of the modified distortion multiple description problem is known, which is 
given by (fT2)) . Particularly, the expressions of the optimal nj(e) (uj is the scalar version of Nj 
in (fT2l)). j = 1, . . . , J take the form (see Proposition 12.61) 

2 

Taking the limit of e, i.e. e | 0, there are do{e) I do, aj{af{e)) t cr^ and n j(e) f +00. Due to the 
limiting operation nj(e) f +00, the side distortion d'^ does not contribute to the resulting optimal 
weighted sum rate (fT2)) . In other words, the optimal weighted sum rate (fT2)) is only a function 
of {di, . . . , kL_i, do). Thus, we show that the outer bound (fT2l) is still tight for the original 
multiple description problem with distortion constraints (di, . . . , di, do) (i.e., the individual side 
distortions are loose) . 

In summary, for a scalar Gaussian source, the Gaussian description scheme fully characterizes 
the rate region. When the central distortion constraint is trivial (k^ > M^), the shape of the rate 
region belongs to the class of contra-polymatroids [|T4l|. If it is not the case, one can always 
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construct an optimal Gaussian test channel satisfying the distortion constraint. This is to say that 
when < k^, there always exists < aiaj <,...,< ajaj < aja^ such that 



ai2^[<yi+h) = — 2 2' (52) 

i=l ''^l '^2 ~ '^1 

rrij 

E(-| + ^M^-^-^r' + ^ - ^^F^ = ^ - J = 2, . . . , J - 1, (53) 

A + «7 + fcM/-+.)-^ + ^ - = "^('^^ + ^0 )-^ (54) 



X{(tI -(Tj) = 0, A > 0, (55) 



where 



and /cz, / = 1, . . . , L are as defined in gSJ). The conditions (l52l)-(l55]) follow from (139l)-(|40l), (145]) 
and (|47l) . Further, the solution to (|52l)-(|55l) is unique. 

The solution to (|52l)-(|55]) can be interpreted as a solution to an optimization problem. We 
first introduce parameters yj > 0, j = 1, . . . , J where cr| = J2i=iyi- Define a new function 

Fivi, ...,yj) as 

/ J \ J ™i / J \ .1-1 

V rf=l / J = l i=l \d=l / j=l 

- - aj+i)log I + "2 logyi - aj-i \og(^yi). (56) 

Consider the following optimization problem: 

{0 < Uj, j = 1, . . . , J 
(57) 

Using the concept of Lagrange duality [fTSl Chapter 5] to handle the constraint J2j=iyj — 
the corresponding Karush-Kuhn-Tucker (KKT) conditions take the form 

dF 

— -7 = 0, j = l,...,J (58) 

J 

yj < al and > 0, j = 1, . . . , J. (60) 
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One can show that (|58l)-(l60l) are equivalent to (|52l)-(|55]) by plugging yj = cr| — a'j_i and 7 = A 
into the expressions. Thus, instead of solving a set of equations, we can equivalently consider 
an optimization problem. The sufficient condition making the KKT conditions hold is A;^ < 
as discussed in Scenario 2. 

VI. Conclusion 

We have addressed the rate region of the multiple description problem with respect to 
individual and central distortion constraints for a vector Gaussian source. An outer bound was 
derived for the considered rate region. In the special case of a scalar Gaussian source, the lower 
bound was shown to be tight. 

The work in [jSJ treated the special case that all the weighting factors in ([3]) are identical (i.e., 
J = 1), which corresponds to minimizing the sum rate. In this particular case, it was previously 
shown that the outer bound given by (fT3] ), remains tight even if some of the distortion constraints 
are loose. Thus, the optimal sum rate for individual and central receivers has been completely 
characterized [|8l. While we have not been able to prove similar result for the weighted sum rate 
case (where J > 1), we believe that the outer bound given by (fT2l) . remains tight even if some 
of the distortion constraints are loose. 

Appendix A 
Useful Matrix Lemmas 

Lemma A.l (Matrix Inversion Lemma): [[T6l Theorem 2.5] Let A be an m x m nonsingular 
matrix and B he an n x n nonsingular matrix and let C and D he m x n and n x n matrices, 
respectively. If the matrix A + CBD is nonsingular, then 

(A + CBD)^ = A-^ - A-^C{B-^ + DA^C)^DA^. (61) 



Lemma A.l: [fT6l Theorems 6.8 and 6.9] Let A and B he positive definite matrices such that 
A ^ S (A ^ S). Then 

1^1 ^ \B\ {\A\ h \B\) 
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(62) 



Appendix B 
Proof of Lemma [3]T] 



The main tool used in the proof is the generalized Costa's entropy-power inequality (EPI) 

developed by R. Liu et al. in [fTTl . Specifically, we use the conditional version of the generalized 

Costa's EPL which was shown to be a simple extension of the generalized Costa's EPI [17J. To 

make the work complete, we present the result in a lemma below. 

Lemma B.l (Theorem 1, Corollary 1 in fTT^): Let 2; be a Gaussian random n-vector with a 
positive definite covariance matrix C, and let A be an n x n real symmetric matrix such that 
^ A ^ /„. Then 



exp 



-h{p + A^z\v) 

n 



>\I- A|" exp 



-h{p\v) 
n 



exp 



-h{p + z\v) 
n 



(63) 



for any v and n-vector p independent of z. The equality holds if (p, v) are jointly Gaussian 
with a conditional covariance matrix B = Cov[p|v] such that B — AB and B + A2CA2 are 
proportional. 

Next we are in a position to prove the lemma. The basic idea is as follows, we first apply 

a linear transformation to the raA^-dimensional random vectors a;", and Z2 such that after 

transformation the random vectors corresponding to 2" and have diagonal covariance matrices. 

The Costa's EPI (|63l ) is then used to prove an extremal entropy inequality in the transform 

domain. Finally, the upper-bound inequality (fT9l) is obtained by converting the extremal entropy 

inequality to the original domain. 

We consider diagonalizing Ni and N2 simultaneously. Using the fact that A^i and N2 are 
positive definite, it is known ||T8l that there exists an invertible matrix U such that 



U^NxU = Al and U*N2U = A2, 



(64) 
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where Ai and A2 are positive definite diagonal matrices. From the assumption on Ni and N2, 
it is immediate that Ai -< A2. Let 



and = (-fn®C/*)(a;" + 2^). 



(65) 
(66) 



Note that by applying the linear transformation in (|65l) - (|66l) . the resulting random vectors 
corresponding to and Z2 have independent components. To simplify the notation in the 
derivation, we introduce 



and 5" = (/„ » U')z^. 



(67) 
(68) 



Consequently we have Cov[z^ 



A2. The two variables and y2 can be equivalently 



written as 



(69) 
(70) 



where A = AiA2^ satisfies -< A -< /. Considering the quantity h{y^\v), it is immediate 
from Lemma IB. II that 



h{y^\v) >^log 



A I « exp 



Nn 



\A\ « exp 



Nn 



Hy'i\v) 



(71) 



We now prove an extremal entropy inequality in the transform domain. By using (1711) . we 
have 



fi2h{y^\v) - ^llh{y'l\v) + (^1 - /i2)/i(&"|i;) 



< ^i2h{y'i\v) log 



/at — A| n exp 



log 



IN 



A| " + \ A\ « exp 



A\ N exp 
2 



Nn 



I(z":x' 



(72) 



June 16, 2010 



DRAFT 



26 



From (1721 ). the function 



f {t)=f,,t-^^ log { \I N - + \A\^exp 



Nn 



(73) 



is concave in t and has a global maxima at 



2 V fJ.1 - fJ.2 



(74) 



Combining (|72|) and (1741) produces 



M2/i y^k - Mi/i + 0^1 - M2 /'^ < ^ log , ^ ' .^..'i - ^ log , ^ ' %^|."| 

2 (/ii - ^2)^^ |Ai| 2 (^1 - /i2)^^|A2| 



(75) 



The equality in (l75l) holds if (5;", i;) satisfies the equality condition in (TtTI) imposed by the 
generalized Costa's EPI and the optimality condition (l74l) . From Lemma IB.ll the equality 
condition to (TtTI) can be mathematically written as 



(76) 



where c is a scalar variable. By combining ([741) and (1761) . it is found that the above two conditions 
are equivalent to the fact that (i", v) are jointly Gaussian with a conditional covariance matrix 



Cov[x"\v] = J„ (g) [(mi - /i2)A2(M2A2 - MlAl) ^Ai] . 



(77) 



The final step is to convert the inequality (1751) to an inequality in the original domain. In other 
words, we relate (|75])-(177]) to ([I9l)-(l20l). From (I65I)-(I67]), we arrive at 

h{y':\v)^h{x'' + z1\v)+nlog\U% i = 1,2 (78) 
h{x''\v) = h{x''\v) + nlog\U*\ (79) 
Cov[x'>] = [/„ ® t/*] Cov[a;"|t;] [/„ (g) t/] . (80) 

By plugging (l78])-(l80l) into the derivation of ([l9l)-(l2Ql) is then straightforward. The 

condition -< HiNi -< 112^2 imposed on Ni and N2 is to ensure that the conditional 
covariance matrix Cov[a;"|i>] in (|20|) is positive definite. 
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Appendix C 
Proof of Lemma [24] 



The proof is essentially the same for every j = 1, . . . , J. Thus we only consider the derivation 
for a particular j. we first let Aj >- 0. By using Lemma IATI there is 



^ + {In: ■ • ■ , In)k Mi}^^'^' ■ ■ ■ ' 



Aj - Aj{In, ...Jn) 



K 



{I,.. .Mi} 



+ {In,.--JnYAj{In,...Jn) {In,---,InYA 



■Aj Aj 



{In:---Jn) i^K^^ _ j^j,-iy+H(^Ajj {In,.-.,InY 



i=l 



By rearranging the items in (|8T| ). we have 



{In, ■ ■ ■ ^^n)K ^^ jy_jjyilN, ■ ■ ■ ,InY 



Aj-A,(^{In,...,In)[k 



H®Aj) {In,...JnY + 



A,+ { {lN,...,lN)iK 



{i,...3/r } 

In -1 



H^AA {In,...,InY + 



K 



With the definition of K q in ([8]), (|82l) can be rewritten as 



H ® AA {In, InY+J2 i^Mr'+^ + ^ 



(81) 



(82) 



(83) 



We now consider simplifying (l83l) further. By using Lemma IaTTI there is 



i-Aj) ' + ilN,...,lN)[K^^^ ^^j,-iy+H^Ajj {In,...,InY 
= -A, - i-A,) {In, ...,In) K-1^^^^^,_,^{In, InY{-A.j). 



(84) 
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Similarly to the derivation of (f82l ) from (|8T1) . (I84l) can be rewritten as 

(JAr,...,7jv) (i^^i +i? (g) (/at,..., Jat) 

-Aj- - Aj {In, . . . , /at) -'^{"i^....Mr^}(^^' • ■ • ' ^^y^- 



K 



■{I,.. .Mr'} 



A 



(85) 



Combining (|83l) and (f85l) produces (flTl) . For the case that Aj y 0, there exists 5 > such that 
+ c/at :^ and the corresponding K^^ :^ for e G (0, 5), which supports the previous 
argument. By letting e — )■ 0"*", we obtain the result. The proof is complete. 



Appendix D 
Proof of Lemma [STTI 

We prove the lemma using induction argument. We first consider the monotonicity properties 

of cr| and k^^ j^jij over af, referred to as Case 1. Later, we extend the analysis to the general 

case. The proof is rather long. We present the proof in several steps. 

Case 1 (j = 1): In order to discuss the properties of (t| and k^-^ j^^i-^ as functions of af, we 
first study the support region of af such that a| > crj. By combining (|39|) and (l40l) . we obtain 



a2 , a2 — ai 



aiaf ai{ai-af) .^^ 



In order that > erf, there is 



aiaf ^ ^ erf + fc^ ai 



mi 7 / 



Letting 

mi 

z— 1 ^ ^ 

we have /(O) > and /(+oo) < 0. By using intermediate value theorem |[T9l p. 48], it is 
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obvious that there exists (T|^| > such that 

< (Ti < cr| > CTi (87) 

{fjo -> (T? as (T? 
(88) 
o-f ^- +00 as erf ^ (T|^| 

Further, by analyzing (|39l)-(l40l) under the situation (f87])-(|88l). we find that ^{i_,..^mi} be 
bounded as ^ 

where Ai > 0, and is determined by a'^^y. The key observation here is that ... is bounded 
away from and +oo. The particular value Ai is not important. Using the fact that k^-^ > 
and (l40l) . it is straightforward that aiaf < a2cr|. 



We now discuss the monotonicity properties of and cr| over af E {0,a'^^y] 

Calculating the differentiation of (|39l) and (|40l ) with respect to af produces 



(90) 



Combining Q9} and (|90l) yields 

dfc{i,...,Mi} 



(92) 



dkf 1 71 r 1 1 J 2 

It is easily seen from (EB-dlH) that ^^^f' < and ^ > when e (0,^1^). Thus, 
^{i,...,M}} monotonically decreasing over af E (0,(T|^|), and bounded away from and +oo. 
The parameter (j| is monotonically increasing over af E (0, and can take any positive real 
value. The parameters af and (t| satisfies the inequality aiaj < a2o\ for any o\ E (0,(j|j^i.). 

Case 2 (j = 2, . . . , J — 1): We now study the monotonicity properties of (y']j^\ and k^^^ j^jy 
over af by assuming that some prior information about cr| and fc^^ aP'^} known. Specifically, 
we assume that (t| is monotonically increasing over af E (0, (5"|^ j-i})' '•^^^ ^'^y positive 

real value. The parameter is determined using the constraint that cr| > crj_i. Also we 
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assume k^-^ m^-^} monotonically decreasing over af G (0, (T|^ j-i})' bounded by 



Similarly to the analysis for the case of j = 1, we first study the support region of af such 
that cr|^i > (t|. Again by combining (I39l)-(l40l). there is 



We consider the inequality 



^3 i— 1 



^ CT^' + fcMr^+^ '^i + ^{i,...,A/r'} 

Note that (j| and k^^ m(~'^} varying simultaneously along with af. Thus a direct extension 
of the analysis for Case 1 is unapplicable here. Fortunately, it is known from the assumption 
that k^-^ A/p^} bounded away from and +oo when E (0, a"^^ j-i})- using the above 
observation and the intermediate value theorem as in Case 1, we conclude that there exists 
^ -^^ such that 



< ^{1,...,,} < ^{1,...,-!} 



0<al< a^i,...,,} ^ fff+i > a] (94) 



and < ' (95) 

The analysis implies that when the constraint cr|^^ > crj is imposed, the support region of al 
such that (t| > crj_^ becomes narrow, but still exists. Again by studying (I39l)-(l40l) under the 
situation (|94| ) and (|951) . we arrive at 

<fc{i....,A/^}< I^E^r'j , (96) 

where Aj > 0, and is determined by (T|^ jy Using the fact that k^-^ ^^j^ > and (|40l) . we 

obtain Q;jcr| < aj+io^ for any cr^ G (0,(7|-,^ ^.|). 

Next we consider the monotonicity of a'jj^^ and Af^+^} ^^^^ "^i ^ (^j '^{i j})- Calculating 
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the differentiation of (l39l)-(|40l) w.r.t. af produces 

[-^^ + 4 ) (^u,....M^ 1 + -1) = E 4 (-f + fcMr^+ 

and - ^=i^^^fc-' - ^^j' "j+i I ^^^'+1 (98) 

dcr2 {i,...,A/^} dafiaj - aj+i)cr^ daj {a, - aj+i)aj^i' 

Similarly to that of Case 1, By combining (l39l) and (|971) . we have 

-1 / \ -1 



-2 



1=1 
2 n 



(99) 



Using the prior information that (j| is monotonically increasing over af G (0, cr"^^ jy) and 
— {1 Af^~^} monotonically decreasing over af E (0, (T|^ ^|), and (|98l) -(l99l). we arrive at 



dfc 



^^daf^^ ^ ^ ^^'^ ^ ^ ^ ^^^^ ^1 ^ "^{1 j})' Thus, the monotonicity properties of k^^ j^^y 
and (t| are proved. 

The above induction argument implies that (T|^| > (T|^ > . . . ,> (3"|^ > 0. Thus when 

o-f E (0, (T|^ it is guaranteed that (|4T1) holds. The inequality (|4TI) plays an important role 

in showing that the Gaussian test channel achieves the optimal sum rate. 

Case 3 (monotonicity of A;^ (or ^{i ... ^^})): We list the prior conditions for the derivation. From 
the analysis in Case 1 and 2, it is clear that k^-^^ m-^-^} i^ monotonically decreasing over af E 
(0, (T|-^ and bounded away from and +oo, i.e. A j_i < k^-^ m/^^} < (Yji=i ■ 

Also (Tj is monotonically increasing over af E ^0, (T|^ ' ^^'^ '^^^ take any positive real 

value. 

We work with (|39l) . In order that fcg is positive, we consider the inequality 
Again using the fact that k^^ i^ bounded away from and +oo, and the intermediate 
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value theorem, we conclude that there exists < < a'^^ such that 

< CT? < CT? ^ fc^ > (101) 

and (i.-(E.t.V)"-'f-V 002, 

fcu, — >■ as af — > af 

To show that is monotonically increasing over af E (0, af), one can take the differentiation 
of (|39l ) w.r.t. cr^ and use the monotonicity properties of aj and m-'-I}- The argument is 
straightforward. The proof is complete. 



Appendix E 
Proof of Lemma [2751 

To prove that the corresponding is positive definite, it is sufficient to show that 
> 0, V j = 1, . . . , L. Before presenting the proof, we first show that ^p^, j = 

1, . . . , J are positive definite. Using the fact that -< aiAi -< «2^2(^i) -<•••-< ajAj(Ai) 
and (fT6l) . we arrive at 

Kr, -< K{i,,„M,'-'} ^ ■ ■ • ^ K{l.,...,Ml}- (103) 



We now use induction argument to prove the lemma. Assume K^-^ 0' we show that 

the determinant of K^-^ A/p^+i} positive for any i = 1, . . . ,mj. For the special case that 
j = 1, we make no assumption. The argument is provided as follows: 



K 



{l....,M(-^+i} 



{!,... ,Mi-^} + H Aj, K + Aj, K + Aj'j - {In,...,InYAj{In,...,In) 



{In,---,In) Aj{lN,--.,lN) -drngl^ {^K^^ _j^jj-iy+H'»Aj 



(a) 



i 

n \KMi 



^{i,...,Mr^} 



fc=l 



fe=l 



fc=l 



(104) 
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where (a) follows from (|85] ) and the determinant identity \Im — AmxnBnxml = \In 
BnxmAmxnl- Since K Mj'~^} assumed to be positive definite, it is straightforward that 

■^{i M^'^} + H 1^ Aj > 0. From the condition of the lemma, it is also known that | Aj| > 
and + Aj\ > 0, k = 1, . . . ,mj. Now we prove that the remaining quantity in (11041 ) 

is positive. From (flTI) . there is 



k=l 

fc=i 



/£=1 



where (a) follows from (|103l) and Lemma IA.2I It is immediate that \Aj'^ — 
Y^l^^ ^K^^j-i^^ + Aj^ + ^^j-ij + j I > 0. Thus we have proven that the 

determinant of K^^ Afp^+i} positive for any i = 1, . . . ,mj, j = 1, . . . , J. We conclude 
that the considered is positive definite. The proof is complete. 
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